Super-resolution method using local binary pattern classification and linear mapping

ABSTRACT

A super-resolution method according to the inventive concept may include receiving an input image of low resolution, separating the input image into low resolution (LR) unit patches, classifying a texture type of each of pixels included in the input image using a local binary pattern for the LR unit patches, generating high resolution (HR) unit patches corresponding to each of the pixels based on a mapping kernel corresponding to the texture type, and combining the HR unit patches based on a predetermined setting to generate an output image of high resolution.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of International PatentApplication No. PCT/KR2019/003764, filed on Apr. 1, 2019, which is basedupon and claims the benefit of priority to Korean Patent Application No.10-2018-0040419, filed Apr. 6, 2018. The disclosures of the above-listedapplications are hereby incorporated by reference herein in theirentirety.

BACKGROUND

Embodiments of the inventive concept described herein relate to asuper-resolution method, and more specifically, to a super-resolutionmethod for classifying a texture using a local binary pattern and usinglinear mapping through a mapping kernel corresponding to the classifiedtexture, thereby increasing resolution.

Various types of electronic devices have been developed and distributedbased on electronic technology. In particular, a display device such asa TV, which is one of the most widely used home appliances in generalhomes, has rapidly developed in recent years. As performance of thedisplay device is advanced, a lot of research has been conducted toimprove image quality of the display device.

In addition, as a 4K-UHD TV market is greatly expanded, demand for UHDcontent has also rapidly increased. However, the UHD content is still ina very short supply compared to Full-HD content. Meanwhile, aconsiderable cost is required to produce the UHD content. Therefore,there is a need for a technology to convert the existing Full-HD contentinto UHD quality. However, the conventional resolution enhancementtechniques have a disadvantage that requires a lot of time to convertthe resolution (size and sharpness of an image).

PRIOR TECHNICAL DOCUMENT Patent Document

-   -   (Patent Document 1) Korean Patent Publication No.        10-2015-0107360

SUMMARY

Embodiments of the inventive concept provide a super-resolution methodfor quickly converting a low resolution input image into a highresolution output image in real time.

According to an embodiment of the inventive concept, a super-resolutionmethod may include receiving an input image of low resolution,separating the input image into low resolution (LR) unit patches,classifying a texture type of each of pixels included in the input imageusing a local binary pattern for the LR unit patches, generating highresolution (HR) unit patches corresponding to each of the pixels basedon a mapping kernel corresponding to the texture type, and combining theHR unit patches based on a predetermined setting to generate an outputimage of high resolution.

As an embodiment, the receiving of the input image may includeconverting an original image into a YUV format and then selecting aY-channel image.

As an embodiment, each of the LR unit patches may include one centralpixel and peripheral pixels surrounding the central pixel.

As an embodiment, the classifying of the texture type may includecalculating a difference between a pixel value of each of the peripheralpixels and a pixel value of the central pixel to generate the localbinary pattern and converting the local binary pattern into a decimalnumber to determine a unit patch grade of the central pixel.

As an embodiment, the generating of the local binary pattern may includematching the associated peripheral pixel to 0 when the pixel valuedifference has a negative number, matching the associated peripheralpixel to 1 when the pixel value difference has a positive number, andarranging 0 or 1 corresponding based on a set order of the peripheralpixels to generate the local binary pattern.

As an embodiment, the mapping kernel may be determined based on the unitpatch grade.

As an embodiment, the mapping kernel may be determined based on anequation below, and in the equation below, “y^(l)” may be a vectorizedLR unit patch, “y^(h)” may be a vectorized HR unit patch, and “λ” may bea weight.

$\begin{matrix}{M_{i} = {{\underset{M \in ^{D \times D}}{\arg \mspace{11mu} \min}{\sum\limits_{j = 1}^{K}{{y_{j}^{h} - {My}_{j}^{l}}}_{2}^{2}}} + {\lambda {M}_{F}^{2}}}} \\{{= {{\underset{M \in ^{D \times D}}{\arg \mspace{11mu} \min}{{Y_{i}^{h} - {MY}_{i}^{l}}}_{2}^{2}} + {\lambda {M}_{F}^{2}}}},}\end{matrix}$

As an embodiment, the mapping kernel may be determined based on anequation below, which is calculated by applying least squareminimization to the equation, and in the equation below, “I” may be anidentity matrix, and “Y^(T)” may be a transpose matrix of “Y”.

M _(i) =Y _(i) ^(h) Y _(i) ^(l) ^(T) (Y _(i) ^(l) Y _(i) ^(l) ^(T)+λI)⁻¹

As an embodiment, the generating of the HR unit patch may includeconverting each of the LR unit patches into an LR unit matrix accordingto a first rule, performing dot product calculation of the LR unitmatrix and the mapping kernel to generate an HR unit matrix, andconverting the HR unit matrix into each of the HR unit patches accordingto a second rule.

As an embodiment, the LR unit patches may include overlapping pixelswith one another.

As an embodiment, a central pixel of a first LR unit patch may bedifferent from a central pixel of a second LR unit patch.

As an embodiment, each of the HR unit patches may be disposed around acentral pixel of each of the LR unit patches corresponding thereon.

As an embodiment, the HR unit patches may be arranged adjacent to oneanother at the same interval to constitute the output image.

As an embodiment, each of the LR unit patches may include 3×3 pixels andinclude one central pixel and eight peripheral pixels surrounding thecentral pixel.

As an embodiment, each of the HR unit patches may include 2×2 pixels.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and features will become apparent from thefollowing description with reference to the following figures, whereinlike reference numerals refer to like parts throughout the variousfigures unless otherwise specified, and wherein:

FIG. 1 is a schematic diagram illustrating a super-resolution methodaccording to an embodiment of the inventive concept;

FIG. 2 is a flow chart illustrating a super-resolution method accordingto an embodiment of the inventive concept;

FIGS. 3 and 4 are diagrams illustrating a relationship between an LRunit patch and an HR unit patch;

FIG. 5 is a diagram illustrating a method of determining a grade of acentral pixel included in an LR unit patch; and

FIG. 6 is a diagram illustrating a method of determining an HR unitpatch from an LR unit patch.

DETAILED DESCRIPTION

It should be understood that both the foregoing general description andthe following detailed description are exemplary and it should beconsidered that an additional description of the claimed inventiveconcept is provided. Reference numerals are indicated in detail in thepreferred embodiments of the inventive concept and examples of which areindicated in the drawings. Whenever possible, the same referencenumerals are used in the description and drawings to refer to the sameor similar parts.

Terms such as “first” or “second” may be used to describe variouselements but the elements should not be limited to the above terms. Theabove terms are used only to distinguish one element from another. Forexample, a first element may be referred to as a second element withoutdeparting from the scope of rights of the inventive concept, andlikewise a second element may be referred to as a first element.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it may be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, it will be understood that when an element isreferred to as being “directly connected” or “directly coupled” toanother element, there are no intervening elements present. Meanwhile,other expressions describing relationships between elements such as“^(˜)between”, “immediately ^(˜)between”, or “directly adjacent to^(˜)”may be construed similarly.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the inventiveconcept. The singular forms “a”, “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises,”“comprising,” “includes,” and/or “including,” when used herein, specifythe presence of stated features, numbers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, numbers, steps, operations, elements,components, and/or a combination thereof.

Unless otherwise defined, all terms used herein, including technical orscientific terms, have the same meaning as commonly understood by one ofordinary skill in the art to which this inventive concept belongs. Itwill be further understood that terms, such as those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted to have ideal or excessively formal meaningsunless clearly defined herein.

Hereinafter, embodiments will be described in detail with reference tothe accompanying drawings. However, the scope of the patent applicationis not limited or limited by these embodiments. The same referencenumerals in each drawing indicate the same members.

FIG. 1 is a schematic diagram illustrating a super-resolution methodaccording to an embodiment of the inventive concept. Referring to FIG.1, the super-resolution method of the inventive concept may be performedthrough a local binary pattern (LBP) type classification unit 100 and alinear mapping unit 200. The super-resolution method of the inventiveconcept may increase resolution of an input image (e.g., N×N resolution)to generate an output image (e.g., 2N×2N resolution). The input imagemay be referred to as a low resolution (LR) image, and the output imagemay be referred to as a high resolution (HR) image. For example, thesuper-resolution method of the inventive concept may convert a Full-HDimage into a 4K UHD image or a 4K UHD image into an 8K UHD image.

In order to perform the super-resolution method of the inventiveconcept, an original image may be converted from an RGB format to a YUVformat. A Y-channel (luminosity) image may be selected as the inputimage. A U-channel image and a V-channel image may be converted using aBicubic interpolation method.

The input image may be input to the LBP type classification unit 100.The LBP type classification unit 100 may classify the input image intolow resolution (LR) unit patches. Each LR unit patch is set as a groupof 3×3 pixels included in the input image. Each of the pixels may beexpressed as a pixel value (luminosity). However, the LR unit patch isnot limited thereto, and the number of pixels included in the LR unitpatch may be variously set.

The LBP type classification unit 100 may classify a texture type of eachpixel included in the input image using a local binary pattern for theLR unit patches. For example, each of the LR unit patches may includeone central pixel and eight peripheral pixels. The LBP typeclassification unit 100 may determine a texture type of the centralpixel of each of the LR unit patches. The LBP type classification unit100 may calculate a difference between a pixel value of each peripheralpixel and a pixel value of the central pixel. Depending on a calculationresult, the pixel value difference may have a negative number (−) or apositive number (+). The LBP type classification unit 100 may match theassociated peripheral pixel with 0 when the pixel value difference hasthe negative number. The LBP type classification unit 100 may match theassociated peripheral pixel with 1 when the pixel value difference hasthe positive number. That is, the eight peripheral pixels may correspondto 0 or 1, and each of the LR unit patches may be represented in binary.The binarized LR unit patches may be converted into decimal number andmay correspond to one of 256 grades. That is, the texture type of thecentral pixel of the LR unit patches may be classified based on the 256grades. The method of classifying texture types of central pixelscorresponding to the LR unit patches using a local binary pattern willbe described in detail with reference to FIG. 5.

When the texture type is classified for the central pixel of the LR unitpatches, the linear mapping unit 200 may apply a mapping kernel to eachof the LR unit patches to generate high resolution (HR) unit patches.For example, the linear mapping unit 200 may arrange the LR unit patchesin a specific order to generate a 1×9 LR unit matrix. The linear mappingunit 200 may multiply the LR unit matrix by the mapping kernel tocalculate a 4×1 HR unit matrix. That is, the mapping kernel may have asize of 9×4. The size of the mapping kernel may be determined dependingon a size of the LR unit matrix and a size of the HR unit matrix. Thelinear mapping unit 200 may convert the HR unit matrix into a 2×2 HRunit patch. The linear mapping unit 200 may combine the generated HRunit patches to generate an output image. An operation method of thelinear mapping unit 200 will be described in detail with reference toFIG. 6.

FIG. 2 is a flow chart illustrating a super-resolution method accordingto an embodiment of the inventive concept. FIGS. 3 and 4 are diagramsillustrating a relationship between an LR unit patch and an HR unitpatch. FIG. 5 is a diagram illustrating a method of determining a gradeof a central pixel included in an LR unit patch. FIG. 6 is a diagramillustrating a method of determining an HR unit patch from an LR unitpatch.

Referring to FIG. 2, a super-resolution method of the inventive conceptmay include receiving an input image of low resolution in S100,separating the input image into LR unit patches in S200, classifying atexture type of each pixel included in the input image using a localbinary pattern for the LR unit patches in S300, generating HR unitpatches corresponding to each of the pixels included in the input imagebased on a mapping kernel corresponding to the texture type in S400, andcombining the generated HR unit patches to generate an output image ofhigh resolution in S500.

In S100, an original image may be converted from a RGB format to a YUVformat. A Y-channel (luminosity) image may be selected as the inputimage. Accordingly, each pixel included in the input image may beexpressed as a pixel value (luminosity).

In S200, referring to FIGS. 3 and 4, the input image may be separatedinto LR unit patches. For example, each of the LR unit patches mayinclude 9 pixels as a 3×3 unit patch. Each of the LR unit patches mayinclude one central pixel and eight peripheral pixels positioned aroundthe central pixel. In the LR unit patches, the central pixel may be setas moving side by side. For example, in FIG. 4, the LR unit patches maybe set as LR unit patches LRUP1, LRUP2, and LRUP3. That is, different LRunit patches may include overlapping pixels.

In S300, referring to FIG. 5, a unit patch grade of each of the LR unitpatches may be determined based on pixel values of pixels included ineach of the LR unit patches. For example, in FIG. 5, a pixel value ofthe central pixel may be expressed as IC and pixel values of peripheralpixels may be expressed as I1 to I8. A pixel value difference obtainedby subtracting the pixel value IC of the central pixel from each of thepixel values I1 to I8 of each peripheral pixel may be expressed as anegative number (−) or a positive number (+). When the pixel valuedifference has the negative number, the associated peripheral pixel maybe corresponded to 0. When the pixel value difference has the positivenumber, the associated peripheral pixel may correspond to 1. That is,the eight peripheral pixels I1 to I8 may correspond to 0 or 1,respectively and each of the LR unit patches may be binarized andexpressed as a local binary pattern. The local binary pattern may beconverted into decimal number. The decimal number is defined as the unitpatch grade. Each of the LR unit patches may correspond to one of 256unit patch grades. That is, each of the LR unit patches may have atexture type classified based on the unit patch grade.

In S400, referring to FIG. 6, the LR unit patches may be arrangedaccording to a specific rule and converted into a low resolution (LR)unit matrix. For example, the LR unit patch may be converted into a 1×9LR unit matrix. Dot product calculation of the LR unit matrix and themapping kernel may be performed. The mapping kernel may have a size of9×4. The size of the mapping kernel may be determined depending on asize of the LR unit patch and a size of the HR unit patch to begenerated.

The mapping kernel may be determined based on the unit patch gradedetermined in S300. The mapping kernel may be specified based on theunit patch grade. For example, the mapping kernel may be determined byEquations 1 and 2 as follows.

$\begin{matrix}\begin{matrix}{M_{i} = {{\underset{M \in ^{D \times D}}{\arg \mspace{11mu} \min}{\sum\limits_{j = 1}^{K}{{y_{j}^{h} - {My}_{j}^{l}}}_{2}^{2}}} + {\lambda {M}_{F}^{2}}}} \\{{= {{\underset{M \in ^{D \times D}}{\arg \mspace{11mu} \min}{{Y_{i}^{h} - {MY}_{i}^{l}}}_{2}^{2}} + {\lambda {M}_{F}^{2}}}},}\end{matrix} & \lbrack {{Equation}\mspace{14mu} 1} \rbrack \\{M_{i} = {Y_{i}^{h}{Y_{i}^{l^{T}}( {{Y_{i}^{h}Y_{i}^{l^{T}}} + {\lambda \; I}} )}^{- 1}}} & \lbrack {{Equation}\mspace{14mu} 2} \rbrack\end{matrix}$

When Equation 1 is calculated by applying least square minimization,Equation 1 may be expressed as Equation 2. By solving Equation 2, themapping kernel corresponding to each unit patch grade may be obtained.In Equations 1 and 2, “y^(l)” means a vectorized LR unit patch. “y^(h)”means a vectorized HR unit patch. “λ” means weight and is a scalarvalue. “I” means for an identity matrix. “Y^(T)” means a transposematrix of “Y”. Meanwhile, the mapping kernel may be determined as anoptimal matrix through learning.

The HR unit matrix may be calculated by the dot product of the LR unitmatrix and the mapping kernel. For example, the HR unit matrix may havea size of 4×1. The HR unit matrix may be converted into a 2×2 HR unitpatch according to a specific rule.

Each HR unit patch may be generated based on each LR unit patch.Referring to FIG. 4 again, a first HR unit patch HRUP1 may be generatedbased on a first LR unit patch LRUP1. A second HR unit patch HRUP2 maybe generated based on a second LR unit patch LRUP2. A third HR unitpatch HRUP3 may be generated based on a third LR unit patch LRUP3.

In S500, each of the generated HR unit patches may be arranged at aposition corresponding to a central pixel of the associated LR unitpatch. One central pixel may correspond to a 2×2 HR unit patch. Forexample, referring to FIG. 4, the first HR unit patch HRUP1 may bedisposed around a central pixel of the first LR unit patch LRUP1. Thesecond HR unit patch HRUP2 may be disposed around a central pixel of thesecond LR unit patch LRUP2. The third HR unit patch HRUP3 may bedisposed around a central pixel of the third LR unit patch LRUP3. The HRunit patches HRUP1, HRUP2, and HRUP3 may be disposed adjacent to oneanother not to overlap. Accordingly, an input image of N×N may beconverted into an output image of 2N×2N.

Table 1 below compares calculation times of a LBP-based textureclassification technique of the inventive concept and the conventionalgradient direction-based texture classification technique.

TABLE 1 LBP-based classification Gradient direction-based techniqueclassification technique Super-resolution time 0.016 seconds 43.100seconds

Table 1 shows efficiency of the super-resolution method of the inventiveconcept. Results in Table 1 are results performed on the MATLAB2017bplatform. In Addition, the super-resolution process in Table 1 wasperformed in a PC environment equipped with an Intel i7-7660U dual coreand 16 GB RAM. In Table 1, time taken after 10,000 repetitions wasmeasured using a 3×3 image patch. When the LBP-based textureclassification technique of the inventive concept was used, the timetook 0.016 seconds for the super-resolution, and when the gradientdirection-based texture classification technique was used, the time took43.100 seconds for the super-resolution. That is, the super-resolutionmethod of the inventive concept may be performed at an improved speed ofabout 2,650 times compared to the conventional method.

The LBP type classification unit 100 and the linear mapping unit 200 ofFIG. 1 may be implemented using hardware components, softwarecomponents, or a combination thereof. For example, the device andelements described in embodiments may be configured using one or moregeneral-purpose or special purpose computers, such as, for example, aprocessor, a controller and an arithmetic logic unit (ALG), a digitalsignal processor, a microcomputer, a field programmable array (FPGA), aprogrammable logic unit (PUL), a microprocessor, or any other devicecapable of responding to and executing instructions. The processingdevice may run an operating system (OS) and one or more softwareapplications that run on the OS. The processing device also may access,store, manipulate, process, and create data in response to execution ofthe software. For purpose of simplicity, the description of a processingdevice is used as singular; however, one skilled in the art willappreciated that a processing device may include multiple processingelements and multiple types of processing elements. For example, aprocessing device may include multiple processors or a processor and acontroller. In addition, different processing configurations arepossible, such as parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, for independently orcollectively instructing or configuring the processing device to operateas desired. Software and/or data may be embodied permanently ortemporarily in any type of machine, component, physical or virtualequipment, computer storage medium or device, or in a propagated signalwave capable of providing instructions or data to or being interpretedby the processing device. The software also may be distributed overnetwork coupled computer systems so that the software is stored andexecuted in a distributed fashion. In particular, the software and datamay be stored by one or more computer readable recording mediums.

The methods according to embodiments may be implemented in the form ofprogram instructions that capable of being executed through variouscomputer means to be recorded in computer-readable media. Thecomputer-readable media may include, alone or in combination withprogram instructions, data files, data structures, and the like. Theprogram instructions recorded on the media may be specially designed andconfigured for the embodiment, or may be known to and usable by thoseskilled in computer software arts. Examples of the computer-readablemedia include magnetic media such as hard disks, floppy disks, andmagnetic tape, optical media such as CD ROM disks and DVD,magneto-optical media such as floptical disks, and hardware devices thatare specially to store and perform program instructions, such asread-only memory (ROM), random access memory (RAM), flash memory, andthe like. Examples of program instructions include both machine code,such as produced by a compiler, and files containing higher level codethat may be executed by the computer using an interpreter. The describedhardware devices may be to act as one or more software modules in orderto perform the operations of the above-described embodiments or viceversa.

According to an embodiment of the inventive concept, the low resolutioninput image may quickly generate the high resolution output image inreal time.

According to an embodiment of the inventive concept, the input image ofthe low resolution is separated into the low resolution unit patches,and the low resolution unit patches may be simply classified by gradeusing the local binary pattern.

According to an embodiment of the inventive concept, the low resolutionunit patches which are classified by grade may be quickly converted intothe high resolution unit patches using the corresponding mapping kernel.

As described above, the embodiments have been disclosed in the drawingsand specifications. Although specific terms have been used herein, theseare only used for the purpose of describing the inventive concept, andare not used to limit the meaning or the scope of the inventive conceptdescribed in the claims. Therefore, those of ordinary skill in the artwill understand that various modifications and equivalent otherembodiments are possible therefrom. Accordingly, the genuine technicalprotection scope of the inventive concept should be determined by thetechnical spirit of the appended claims.

What is claimed is:
 1. A super-resolution method comprising: receivingan input image of low resolution; separating the input image into lowresolution (LR) unit patches; classifying a texture type of each ofpixels included in the input image using a local binary pattern for theLR unit patches; generating high resolution (HR) unit patchescorresponding to each of the pixels based on a mapping kernelcorresponding to the texture type; and combining the HR unit patchesbased on a predetermined setting to generate an output image of highresolution.
 2. The super-resolution method of claim 1, wherein thereceiving of the input image includes converting an original image intoa YUV format and then selecting a Y-channel image.
 3. Thesuper-resolution method of claim 1, wherein each of the LR unit patchesincludes one central pixel and peripheral pixels surrounding the centralpixel.
 4. The super-resolution method of claim 3, wherein theclassifying of the texture type includes: calculating a differencebetween a pixel value of each of the peripheral pixels and a pixel valueof the central pixel to generate the local binary pattern; andconverting the local binary pattern into a decimal number to determine aunit patch grade of the central pixel.
 5. The super-resolution method ofclaim 4, wherein the generating of the local binary pattern includes:matching the associated peripheral pixel to 0 when the pixel valuedifference has a negative number; matching the associated peripheralpixel to 1 when the pixel value difference has a positive number; andarranging 0 or 1 corresponding based on a set order of the peripheralpixels to generate the local binary pattern.
 6. The super-resolutionmethod of claim 4, wherein the mapping kernel is determined based on theunit patch grade.
 7. The super-resolution method of claim 1, wherein themapping kernel is determined based on an equation below, and in theequation below, “y^(l)” is a vectorized LR unit patch, “y^(h)” is avectorized HR unit patch, and “λ” is a weight. $\begin{matrix}{M_{i} = {{\underset{M \in ^{D \times D}}{\arg \mspace{11mu} \min}{\sum\limits_{j = 1}^{K}{{y_{j}^{h} - {My}_{j}^{l}}}_{2}^{2}}} + {\lambda {M}_{F}^{2}}}} \\{{= {{\underset{M \in ^{D \times D}}{\arg \mspace{11mu} \min}{{Y_{i}^{h} - {MY}_{i}^{l}}}_{2}^{2}} + {\lambda {M}_{F}^{2}}}},}\end{matrix}$
 8. The super-resolution method of claim 7, wherein themapping kernel is determined based on an equation below, which iscalculated by applying least square minimization to the equation, and inthe equation below, “I” is an identity matrix, and “Y^(T)” is atranspose matrix of “Y”.M _(i) =Y _(i) ^(h) Y _(i) ^(l) ^(T) (Y _(i) ^(l) Y _(i) ^(l) ^(T)+λI)⁻¹
 9. The super-resolution method of claim 1, wherein the generatingof the HR unit patch includes: converting each of the LR unit patchesinto an LR unit matrix according to a first rule; performing dot productcalculation of the LR unit matrix and the mapping kernel to generate anHR unit matrix; and converting the HR unit matrix into each of the HRunit patches according to a second rule.
 10. The super-resolution methodof claim 1, wherein the LR unit patches include overlapping pixels withone another.
 11. The super-resolution method of claim 1, wherein acentral pixel of a first LR unit patch is different from a central pixelof a second LR unit patch.
 12. The super-resolution method of claim 1,wherein each of the HR unit patches is disposed around a central pixelof each of the LR unit patches corresponding thereon.
 13. Thesuper-resolution method of claim 1, wherein the HR unit patches arearranged adjacent to one another at the same interval to constitute theoutput image.
 14. The super-resolution method of claim 1, wherein eachof the LR unit patches includes 3×3 pixels and includes one centralpixel and eight peripheral pixels surrounding the central pixel.
 15. Thesuper-resolution method of claim 1, wherein each of the HR unit patchesincludes 2×2 pixels.